openarenafandomcom-20200216-history
Multithreaded map compiling
It is possible to take advantage of modern multi-core processors to compile maps much faster by using multithreading. A quick way to see if your machine has got multiple cores (or multiple processors), is: * Microsoft Windows: to take a look to the "Performance" tab of the "Task manager" tool (from Ctrl+Alt+Del). If the "CPU utilization history" is divided into two or more graphs, your machine as got at least two or more cores (physical or logical). * GNU/Linux: run the "top" command. After pressing the "1" key, in the upper part of the screen, the "Cpu(s)" line should change into multiple lines ("Cpu0", "Cpu1", "Cpu2"...), one for each core (physical or logical). If you have got more than one core, you may find useful to compile maps (q3map/q3map2) using more threads. This way, you should see higher CPU load in Task Manager/top while compiling, and you should be able to compile maps much faster than using a single thread, because each thread should work on a separate core (with a single thread, only one core would be used). So, choose the number of threads (1, 2, 3, 4, 5, 6...) depending from how many cores you have got and how many of them you want to use for this purpose. Leaving at least one core "free" should allow you to do not experience slowdown using the computer do to other things while you are compiling a map, it's your decision. How to setup multithreaded compiling Q3MAP (map compiler that comes with the original Q3Radiant) and Q3MAP2 (map compiler that comes with GTK Radiant, with some extra features) are command line tools. The map editor launches q3map (or q3map2) with various command-line parameters, depending from the Bsp option you choose from the menu. Q3MAP and Q3MAP2 support the "-threads n" parameter to tell how many threads they have to use ("n" is the number of threads, e.g. 1, 2, etc.). While running on Microsoft Windows OS, Q3MAP2 auto detects number of total cores (physical + logical) your machine has, and automatically selects the number of threads to use accordingly. However you can use the "-threads" parameter to manually force the numer of threads to use. While running Q3MAP2 on GNU/Linux, or while using Q3MAP, instead, you have to manually specify the number of threads to use. Note: q3map2 doesn't know the difference between complete processors, real cores and Hyper-threading technology's logical cores."Hyper-threading" is a technology by Intel. It allows to use multi-threading, but it's not a synonym of "Multi-threading". It'll assume that each "CPU" it finds is a complete CPU. In reality, each physical processor can have one or more physical cores. Each core may have hyperthreading features, that would show it as two cores. So, a dual-core processor with hyperthreading enabled would look like a four-processors machine, but the performances would be lower than a four-processors machine!It's just an example: real multiprocessor machines are usually used only in servers, not in end-user machines. It's unlikely you would buy such hardware to compile OpenArena maps! And a today's high end single processor computer may be better than a ten-years-old double or even quad processors machine. While a dual-core CPU has got two physical cores that share some components, Hyperthreading technology shows an extra (logical) core by simply adding few components to one physical core: depending from the different cases of use, giving load to the Hyperthreading logical core may even hurt performances.This is a generic statement, one should do some tests to verify if it is good or not at compiling maps. There is no advantage in trying to force more threads than the number of total cores you have. While using all your physical dual (or quad, or more) cores should speed up compiling a lot, the effect of hyperthreading (logical) cores is uncertain. If you wish, you can do tests compiling the same map specifying different number of threads, comparing the time needed. Q3Radiant Let's take the case we are using the old Q3Radiant editor, configured for the original Quake III Arena game, on Microsoft Windows. In the "Bsp" menu, you can find the bsp_FullVis (light -extra) option. This is probably the one that takes more time to compile, used also for the "final" map, so it is the one we will tweak. Go to "File" -> "Project settings" menu. In the "Menu commands" section, select bsp_FullVis (light -extra) line and click "Change...". In the window that opens, select all the text in the "Command" field, and copy it to your clipboard (CTRL+C or right button, copy), then click on "Cancel", to be sure we will not damage the original option. Now click on "Add...", and paste from your clipborard into the "Command" field (CTRL+V or right button, paste). Modify from -threads 1 to -threads 2 (or other number, e.g. 3, 4, 5, 6...) in the two places where the "-threads" option appears. Enter bsp_FullVis (light -extra) Multithread (or whatever you prefer, but anyway the name has to begin with bsp, and it should not contain the " character) in the "Menu text" field, then click OK.Alternatively, when you click on "Modify..." after selecting the original menu option, edit the "threads" parameters there and MODIFY the "menu text", then click OK: this will cause the program to automatically create a new menu option, without the need of copy-pasting (we suggest to do not apply any change directly to the original menu options, but to create new options, because you may risk to ruin them, also due to a Q3Radiant bug that may "cut" the command line!). You should have started from something similar to this (of course your Q3 installation path may vary): ! C:\PROGRA~1\QUAKEI~1\Tools\q3map $ && ! C:\PROGRA~1\QUAKEI~1\Tools\q3map -vis -threads 1 $ && ! C:\PROGRA~1\QUAKEI~1\Tools\q3map -light -extra -threads 1 $ And you should have got something like this (in case you want 2 threads): ! C:\PROGRA~1\QUAKEI~1\Tools\q3map $ && ! C:\PROGRA~1\QUAKEI~1\Tools\q3map -vis -threads 2 $ && ! C:\PROGRA~1\QUAKEI~1\Tools\q3map -light -extra -threads 2 $ Still in the "Project Settings" windows, select the newly created option and click "Change...". If everything is still as you set a minute ago, you are ready and you can go compiling your map... but if the line in the "Command filed" appears incomplete (where many options misteriously disappeared), e.g. like this: ! C:\PROGRA~1\QUAKEI~1\Tools\q3map $ && ! C:\PROGRA~1\QUAKEI~1\Tools\q3map you enconountered a bug of the editor (this is another reason we suggest to never modify the original menu options) and thus you have to manually modify the file that contains the menu to fix the menu option (compiling would not run correctly without the missing part of the command line). In this case, close Q3Radiant. Then go to your Quake III Arena installation folder (if you have multiple installations of the game, the one you use with Q3Radiant), then enter the "BASEQ3" folder, and the "scripts" folder in it. There, use a plain text editor (such as notepad) to edit the "quake.qe4" file. Locate the line that contains your newly created menu option (in our example, bsp_FullVis (light -extra) Multithread). As you can guess, the format of a line is simply "menu text" "command". Correct the command line part, copy-pasting from the original menu option and changing the value in the two places of the line where the ''-threads'' option is. Save and close the file, and start Q3Radiant again. At this point, if you select the newly created option from the Bsp menu, you should notice your map compile in less time. Of course, you may have directly edited the quake.qe4 file and added a line (somewhere between the { and } lines) for the new menu option, if you prefer. Also in this case, remember that the name of the new menu option has to begin with bsp and should not contain the " character (that is instead used to separe the menu option name from the command line). Hint: also considering the editor bug, you may want to make a backup copy of that quake.qe4 file before starting to make any changes to it through Q3Radiant or through text editor. GTKRadiant NetRadiant Notes See also * Mapping * Mapping resources & tutorials * Speed up visibility calculations Category:Development Category:Mapping